Regularization in network optimization via trimmed stochastic gradient descent with noisy label

Regularization is essential for avoiding over-fitting to training data in network optimization, leading better generalization of the trained networks. The label noise provides a strong implicit regularization by replacing target ground truth labels examples uniform random labels. However, it can cause undesirable misleading gradients due large loss associated with incorrect We propose first-order optimization method (Label-Noised Trim-SGD) that uses example trimming order remove outliers based on loss. proposed algorithm simple yet enables us impose label-noise and obtain effect than original methods. quantitative analysis performed comparing behavior noise, trimming, algorithm. also present empirical results demonstrate effectiveness our using major benchmarks fundamental networks, where has successfully outperformed state-of-the-art